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(54) Pointing to displayed objects in a computer system 

(57) An object to be pointed to in a user's computer 
display (109) is Indicated with an aninoated icon (113). 
In a computer network, such as the Worldwide Web, a 
pointer is transmitted from one user to another user 
(1 14) via an improved protocol. A graphical design (201) 
of a computer icon is also disclosed. 
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Description 

The present invention relates to computer systenrrs 
and their operation. More specifically, but not exclu- 
sively to a computer system in which objects on a com- s 
puter display are indicated with an improved pointer. 

Various pointer systems for conputer displays are 
well known to those of skill in the art Most commonly, 
such pointers take the form of arrows, l-bars and other 
such elements. In the Windows environment for exanrv 10 
pie. such pointers are moved from one location to 
another by detecting a series of "mouse move" signals 
from a computer pointing device such as a mouse. Gen- 
erally, the computer system monitors inputs from a 
mouse or similar pointing device, and in a roughly con- 15 
tinuous manner, moves a pointer from one screen loca- 
tion to another. The pointer may be used to point to a 
variety of screen objects such as buttons, cursor bars, 
and a wide range of other screen objects. 

In some situations it is desirable for a user to point so 
to an object on a computer display that is on another 
user's screen. As a simple exannple, suppose that a 
user is remotely advising another user regarding a Web 
page design. Commonly, the first user would simply 
advise the second user by verbal communication such 2S 
as through a telephone call or even over an internet 
voice link. It is often awkward or extremely difficult for 
one user to direct another's attention to a specif ic loca- 
tion on the screen display. 

Solutions have been proposed to this problem. For 30 
exanrtple. remote pointing is supported by products such 
as telepointers. Such telepointers may be used to point 
to an object in a screen display, sometimes taking the 
form of a WYSiWIS (What You See Is What I See) dis- 
play. In some cases the displays are maintained in a 3S 
concurrent state by way of a contrdler/controllee rela- 
tionship between the two display^. In such situations, 
the user at a controller display moves his or her pointing 
device, and the pointer on all of the displays of the other 
users is moved in unison. This enables one user to point 40 
to objects and allow others to see where the user is 
pointing. Such products include ShowMe SharedApps® 
from Sun Microsystems®, Inc. See also Stefik et al., 
"WYSIWIS Revised: Early Experiences with Multi-User 
Interfaces." Conference on Computer Supported Coop- 4S 
erative Work. December 3-5. 1986. 

While rriee^g with some success, such systems 
have also ^TwTth certain limitations. For example, in 
some systen^ users are not using the same system for 
rendering their respective displays on their computers, so 
A common example of this situation is found in internet 
browsers where users each select their own internet 
browsing tool, font styles and colors, CPUs, operating 
systems, ard monitors, often with the result that remote 
pointers cannot communicate with each individual user. 55 
Systems such as the Worldwide Web (WWW) are flexi- 
ble through the use of a common HTML image trans- 
mission standard, but do not effectively support remote 
pointing. Further, the communication bandwidth that 



would be required to operate such systems uang con- 
ventional controller/controllee techniques would create 
great burden on any network system Such problems 
would be particularly apparent when users are in differ- 
ent countries, especially if some users have poor con- 
nections to the WWW. 

From the above it is seen that improved systems 
and methods for pointing to objects on computer 
screens are desirable, especially for use in computer 
netwaks. 

Innproved systems and methods for pointing to 
objects on a computer display can be provided by 
embodiments of the present invention. According to one 
specific embodiment of the invention, upon pointing to 
an object, a pointer icon changes appearance. An illus- 
tration of such changing appearance is a "raindrop" icon 
wherein an illustration of a drop of water appears in the 
display and expands over time. 

Ennbodiments may also indude a system for point- 
ing to objects in a computer network. Pointer objects are 
ennbedded in a serial data stream when a first user 
points to an object When the serial data stream is 
assembled on a second user's screen, the display 
includes a pointer at or near the location where ^e first 
user has pointed. An important application of the irrven- 
tion will be found in the Worldwide Web wherein point- 
ers are ennbedded in HTML bit streams of innages. 

Accordingly, one specific embodiment provkjes a 
method of operating a corrputer including the steps of 
moving a pointer from one object to another object on a 
computer screen with a user input device; receiving 
user input for pointing to one of tiie objects on the conv 
puter saeen; and changing an appearance of a pointing 
icon in response to the step of receiving user input for 
pointing to one of the objects. 

An embodiment may also include a method for 
pointing to objects in a computer network including the 
steps of transmitting screen objects across the conpu- 
ter network in a serial form for display on a first user 
screen; receiving pointing input from a second user, the 
pointing input defining a kxation on the screen where 
the second user desires to point; inputting pointing 
objects into the serial stream; and displaying pointer 
graphics on said first user screen corresponding to said 
location where said second user desires to point 

For a better understanding of the invention, and to 
show how the same may be carried into effect, refer- 
ence will now be made, by way of example, to the 
acconnpanying drawings, in which:- 

Fig. 1 is an overall block diagram of a computer sys- 
tem according to one embodiment of the invention 
herein; 

Rgs. 2A to 2G illustrate a dynamic pointer to an 
object on a computer display; the background of the 
screen and the computer rtself are illustrated writh 
broken tines only for purposes of showing the envi- 
ronment of the dynamic pointer and not for the pur- 
pose of illustrating tiie invention herein; 
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Fig. 3 illustrates a method for displaying a pointer 
icon on a coniputer display of a remote user; 
Fig. 4 illustrates a method for generating a particu- 
lar temporally varying pointer icon; and 
Fig. 5 Illustrates a method for transmitting a pointer 
icon from one user to another user in a computer 
network, using the Worldwide Wet as an example. 

Fig. 1 illustrates a computer network according to 
one embodiment of the invention. A computer 101 is io 
provided with a central processing unit 103 and memory 
1 05. Memory 1 05 may include, for example, RAM and a 
permanent storage media such as a magnetic and/or 
optical disks in which the software for generating point- 
ers is stored (as well as other software). The computer if 
101 is coupled to one or more user input devices 107. 
User input devices 107 will typically include a pointing 
device such as a. mouse and a keyboard. The computer 
provides output on display 109. The display will nor- 
mally include one or more screen objects such as but- 20 
tons, regions for user input, pulldown menus, sliders, 
and other objects. A pointer 111 is overlaid on the 
screen display artd can be moved from .one location to 
another on the screen by noanipulation of one or more of 
the user Input devices. Images nnay be generated on the 25 
display via one of a variety of systems such as via a 
Web page. One preferred browser used for this purpose 
is Hot Java, a browser available from Sun Microsystems. 

When the user has moved the screen icon to a 
desired location on the screen, the user provides input 30 
indicating a pointing operation. In some embodiments 
the pointing operation will be performed via a mouse 
click or pressing a designated key on the keyboard. In a 
particular emlxxiiment the operation will be performed 
using a two button mouse in which the left mouse button 35 
is used for more conventional input such as pressing on 
displayed buttons or pointer bars; while the right or alter- 
nate mouse button is operation. 

When the user indicates that he/she desires to 
point to a particular object on the screen, a highlighted ^0 
pointing operation rrtay take place on the user's screen. 
Preferably, the highlighted pointing operation is con- 
ducted by displaying a highlight pointer icon 113 that 
changes temporally such that it will attract the attention 
of users more clearly. In some instances the highlighting 45 
pointer will be displayed for not only the user originating 
the pointing operation, but also others. In such cases, a 
remote^?5ft^5uter system 114 linked by a netwak con- 
nection 1 1 5 is provided with a similar pointer 117 on its 
display. In these cases, one user may point to an object so 
on another users screen with ease. In some instances 
(such as when a computer is used to generate a pres- 
entation) the pointer is displayed only on that particular 
user's display(s). In a preferred embodimertt, each user 
is viewing a Web page generated from an HTML data 55 
stream. Each user will have his/her own browser soft- 
ware, which may or may not be the same browser soft- 
ware. 

Figs. 2A to 2G illustrate the graphical icon design 



utili2ed in a preferred embodiment of the Invention for 
pointing to a screen object A saeen 200 and computer 
200A are illustrated with broken lines in Figs. 2A-2Q 
only for purposes of showing the environment of the 
dynamic pointer and not for the purpose of the invention 
herein. As shown in Fig. 2A a pointer 201 is displayed 
with an appearance similar to that of a drop of water fall- 
ing on the screen. Figs. 2B to 2G illustrate the temporal 
variations in the appearance of the pointer icon. Fig. 2B 
illustrates a portion of the screen before the pointer is 
activated. Fig. 2C illustrates the icon in its initial form. As 
shown, a circular region 203 of "white space* on the 
screen is developed. In Fig. 2D the drcular region 203 ts 
surrounded by a dark region 205. In Rg. 2E the region 
205 is surrounded by a darker region 207. In Fig. 2F the 
initially displayed white region 203 has been removed. 
In Rg. 2G the darker region 205 has been removed. 
Subsequent to the display in Fig. 2G the screen returns 
to its initial form as shown in Fig. 2B. 

Rg. 3 illustrates the overall methodology utilized to 
transmit a pointer from one user to another user accord- 
ing to a preferred embodiment of the inventioa At step 
301 the systerri (which may include, for example, the 
computers of the two users and intervening network 
connputers) inputs a pointing action from a user via, for 
example, a button push on a mouse, input from a touch 
screen display, or from another user input device. At 
step 303 the system (application program, generating 
system, and/or hardware) identifies a closest character 
or object on the screen adjacent to the location where 
the pointing action has taken place. In the HTML con- 
text this step includes identifying the position in the 
HTML stream corresponding to this closest character or 
object. In the event that the user has the pointer 111 
directed to a specific character or screen object this 
object will be chosen as a marked object In cases 
where the user has pointed to a location on the screen 
where no object is displayed, the system will determine 
the distance to a plurality of adjacent screen objects, 
and select the object closest to the pointer location 
when the pointing operation has taken place For exam- 
ple, in the HTML data format of the WWW. the screen 
location is identHied to the system when a user enters a 
button click operatioa The system then calculates the 
distance to each of a plurality of adjacent characters 
that have been generated by virtue of the HTML gener- 
ated display. The pointer icon image(s) are then 
inserted before or after this character in the HTML data 
stream. 

At step 305 the system inserts a pointer element 
into a serial bit stream for transmission of an inwge to a 
remote user. A pointer element may refer to an image, 
object, or even a single indication (like a bold or itaBc 
mark in HTML) that indicates a pointer shouW be <fis- 
played. One approach is to provide an extension to the 
HTML standard that includes a special formatting code 
that indicates a pointer should be displayed. In sonrw 
embodiments, the remote user already has a display 
painted on his/her screen and the system transmits only 
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a series of characters indicating where the existing dis- 
play is to be changed. At step 307 the bit streann with 
the pointer element is transmitted to the remote user. At 
step 309 the remote user's connputer system displays 
the screen image with the pointer icon in place. 5 

In some embodiments the pointer icon's image will 
vary temporally. Accordingly, in such embodiments the 
system will perform a test at step 3 1 1 to d etermine if the 
pointer display operation is complete. If not, the system 
will return to step 305 wherein a revised screen image 10 
will be generated, providing the user with the perception 
that the pointer icon is changing with time (the example 
above being in the form of a drop of water falling on the 
screen). In other embodiments, the transmission of a 
single pointer character or element win cause the 15 
remote user's system to generate the changing pointer 
image locally. For exanple, a pointer indicator may be 
inserted in the HTML data stream arxJ the local browser 
may be programmed to paint a changing icon upon 
receipt of the pointer indicator. so 

Fig. 4 illustrates a process for generating a tenrpo- 
rally varying pointer icon in a particular case of a water 
drop-type d inr\age. As shown, the system first gener* 
ates a white circle 3 pixels in diameter centered at the 
location where pointing is desired at step 401. At step 2S 
403 the system draws an open 50% grey circle sur* 
rounding the white circle. At step 405 the system draws 
an open black circle surrounding the grey drcle. At step 
407 the system removes the white cirde, and at steps 
409 and 4 1 1 the system rennoves the grey and black dr- 30 
cles, respectively. The image displayed according to this 
particular embodiment of the invention will have the 
appearance of a drop of water falling on the screea 

Preferably, the elements displayed according to Fig. 
4 are displayed over a period of about 0.5 seconds 35 
each, and are removed from the screen in another 
group of 0.5 second intervals each. Therefore, the entire 
pointing process takes place over a time span of more 
than 1 second, preferably more than about 2 seconds, 
and most preferably more than 2.5 seconds, but prefer- 40 
ably less than 5 seconds. This is enough time to draw 
the user's attention to the screen location without signif- 
icantly obscuring the display. 

F\g. 5 illustrates the process of generating a pointer 
icon remotely in greater detail, using the Worldwide 4S 
Web as an illustration with the known convention of the 
HTML transmission format The transmission of images 
over the WoiTcfwSe Web is well understood to those of 
skill Ln the art and is described in some detail in Morris, 
HTML For Fun and Profit SunSoft Press. 1995, incor- so 
porated herein by reference for aD purposes. 

At step 501 the system displays a Web page conn- 
mon to both a local and a remote user, or the local user 
controls the remote user Web page. At step 503 the 
system checks to determine if the same Web page is 55 
displayed at both the local and remote user's location. If 
not an error message is displayed. 

At step 505 the system inputs the remote IP 
address of the remote user. At step 507 the system 



optionally activates the pointer system (assuming rt is 
not alv^ys active). At step 509 the system irputs a 
remote pointer action such as a mouse click. At step 
51 1 the system selects a character position in the HTML 
data file for the current Web page closest to that of the 
pointer. At step 51 5 the pointer is transmitted and drawn 
on the remote user display. 

The HTML format is known to those of skill in the 
art. Generically, a Web page includes a stream of bits 
including a header and title, comments, a body, and an 
address. Text is marked with various formatting charac- 
ters such as paragraph format indicators, character for- 
mat indicators, and list format indicators. In^ge, sound, 
and animated video may be inserted into the bit stream 
according to recognized standards. According to a pre- 
ferred embodiment herein, after selection of a tocation 
for the pointer icon, an image of the first "part" of a tem- 
porally varying pointer icon is inserted into the HTML bit 
stream as. e.g., a GIFF, TIFF, or JPEG bitmap, using an 
innage or <IMG>tag. Preferably, transparent cells are uti- 
lized such that the pointer icon may be overlaid on the 
object that is of interest In a transparent display, the 
darkness of the previous screen display and the screen 
display after the addition of the pointer icon are consid- 
ered, tf a pixel in the "underlying" image is dark, it 
makes that pixel darker when the "overtying" pointer 
icon is added. 

The above description is illustrative and not restric- 
tive. Many variations of the invention will become appar- 
ent to those of skill in the art upon review of this 
disclosure. Merely by way of example, while the inven- 
tion has been illustrated primarily with regard to gener- 
ation of "raindrop" pointers over the WaWwide Web. the 
invention is not so limited. 

Claims 

1. A method of operating a conrputer conprising the 
steps of: 

displaying a graphical display on a computer 
screen; 

moving a pointer from one object to another 
object on said computer screen with a user 
input device; 

receiving a first user's input for pointing to one 
of said objects on said computer screen; and 
temporally varying an appeararxe of a pointing 
Icon in response to said step of receiving said 
first user's input for pointing to one of said 
objects. 

2. The method as recited in claim 1 further comprising 
the steps of: 

transmitting a pointer signal to a second user, 
wherein said step of tempaaDy varying an 
appearance of said pointing icon is performed 
on at least said second user's computer 
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saeen. 

3. The method as recited in claim 1 or 2 wherein said 
step o1 temporally varying comprises displaying a 
pointer having an appearance similar to a drop of 5 
water falling on said screen. 

4. The method as recited in claim 1 or 2 wherein said 
step of temporally varying comprises terminating 
display of said pointer icon after a period of time. lo 

5. The method as recited In claim 2, 3 or 4 wherein 
said step of temporally varying an appearance is 
performed on said first user's computer screen. 

6. The method as redted in any one of claims 1 to 5 
wherein said graphical display is generated over a 
computer network on at least said first user's com- 
puter screen and a second user's computer screen. 

7. The method as redted in claim 6 wherein said 
graphical display is a Web page. 

8. The method as recited in any one of daims 2 to 7, 
wherein said step of transmitting a pointer signal 2$ 
comprises the steps of: 

identifying an object in said graphical user 
interface closest to a location of a pointer input; 
and 

inserting a pointing indicator into a serial bit 
stream to be transmitted to said second user. 

9. The method as redted in claim 8 wherein said 
serial bit stream is an HTf^L bit stream. 

10. A method for pointing to objects in a computer net- 
work comprising the steps of: 

transmitting screen objects across said compu- 
ter network in a serial data stream for display 
on a first user screen; 

receiving pointing input from a second user, 
said pointing input defining a location on said 
first user screen where said second user 
desires to point; 

inputting pointing objects into said serial data 
^^^fre^; and 
displaying pointer graphics on said first user 
saeen conesponding to said location where 
said secortd user desires to point 

11. The method as recited in claim 10 wherein said 
serial data stream is an HTML data stream 

12. The method as recited in claim 10 or 11 wherein 
said step of inputting pointer objects into said serial 
data stream comprises the steps of: 
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identifying a screen display object closest to 
said location where said second user desires to 
point; and 

placing said pointing objects into said serial 
data stream at a position in conrespondence 
with said identified screen display object 

13. The method as recited in daim 10, 1 1 or 12 wherein 
said of displaying pointer graphics comprises tiie 
step of temporally varying an appearance of said 
pointer graphics on said first user screen. 

14. A computer system corrprising: 

a computer network for generating screen dis- 
plays on a plurality of user computers, the net- 
work including means for transmitting serial 
data streams to a plurality of user computers 
and means for generating screen displays 
based on said serial data streams; and 
a pointing system comprising a pointer inser- 
tion element for inserting pointers into said 
serial data stream. 

15. A computer system as recited In claim 14 wherein 
said system for transmitting serial data streams is a 
system for transmitting HTML data streams. 

16. A computer system as recited in claim 14 or 15 
wherein said pointing system generates temporaBy 
varying pointer icons. 

17. A conputer system as recited in claim 14. 15 or 16 
wherein said pointing system identifies a closest 
screen object to a pointer, arxl said insertion ele- 
ment selects said closest saeen object in said 
serial data stream for insertion of said pointer ele- 
ment 

18. A graphical design for a temporally varying pointing 
icon in a computer system as shown in Figs. 2B to 

2a 

19. A storage medium conveying software comprising: 

software for generating a pointer in a remote 
display arxj; 

software for temporally varying an appearance 

— of said pointer, whereby a screen location is 

emphasized. 

20. A medium as recited in claim 19 wherein said soft- 
ware for changing an appearance of said pointer 
comprises software for inserting a pointer indicator 
in an HTML data stream. 

21. A medium as recited in claim 20 further conveying 
software for identifying a location of a nearest 
object to a user-defined location. 
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22. A medium as recited in claim 21 further cnveylr>o 
software for inserting a pointing irtdicator adjacent a 
character in a serial data stream adjacent said 
nearest object 

5 

23. A medium as recited in claim 20 further conveying 
browser softwafe associated with said remote dis- 
play which is different browser software from 
browser software from a user location where said 
pointer indicator is generated. io 
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(54) Pointing to displayed objects In a computer system 

(57) An object to be pointed to in a user's conrputer 
display (109) IS indicated with an animated icon (113). 
In a computer network, such as the Worldwide Web, a 
pointer is transmitted from one user to another user 
(114) via an inproved protocol. A graphical design (201) 
of a computer icon is also disclosed. 
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